1. Technical Field
The present invention relates generally to the field of data processing systems and, more particularly, to a method, system and computer program product for fault recovery in a distributed stateful publish-subscribe system.
2. Description of Related Art
A publish-subscribe system is a system that includes two types of clients, publisher clients and subscriber clients. A publisher client, also referred to herein as a publisher, generates messages, also referred to as events, which contain a topic and some data content. A subscriber client, also referred to herein as a client, provides, ahead of time, a criterion, also referred to as a subscription, that specifies the information, based on published messages, that the system is required to deliver to that subscriber client in the future. In a publish-subscribe system, publishers and subscribers are anonymous in that publishers do not necessarily know the number of subscribers or their locations; and subscribers do not necessarily know the locations of the publishers.
A stateless publish-subscribe system, also referred to as a topic-based or content-based publish-subscribe system, is a system in which delivered messages are a possibly filtered subset of published messages, and in which the subscription criterion is a property that can be tested on each message independent of any other message. For example, a filtered published message might be “topic=stock-ticker” or “volume>10000 & issue=IBM”. A stateful publish-subscribe system, on the other hand, is a system without such restrictions. A stateful publish-subscribe system is required to support subscription criteria that depend upon computations that require multiple messages from one or more streams, for example, “Give me the highest quote of IBM within each one-minute period”. In addition, a stateful system might entail delivering information that is more than simply a copy of published messages, e.g. “Tell me how many stocks fell during each one-minute period”.
A stateful publish-subscribe service as used in this invention is implemented on an overlay network that comprises a collection of service machines, also referred to as brokers, that accept messages from publisher clients, deliver subscribed information to subscriber clients, and route information between publishers and subscribers. A stateful publish-subscribe system as used herein is a publish-subscribe system in which at least one subscription of the system is stateful.
An effective stateful publish-subscribe system should be fault-tolerant; i.e., it should have the ability to detect and recover from failures that may occur when a stateful publish-subscribe service is implemented on an overlay network. Such failures may include, for example, temporary crashes of broker machines, temporary losses of connectivity between broker machines, and network errors causing messages to possibly be lost, duplicated or delivered out of order.
There are known techniques in database systems in which each new published event, and the subscriber state changes derived from each new event, can be incorporated into a transaction. An implementation based upon transactions in a database system, however, is inefficient and requires an expensive “two-phase commit” protocol for every message. It would, accordingly, be advantageous to provide a fault-tolerant stateful publish-subscribe system that is efficient and that does not require a two-phase commit protocol for every message.